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ABSTRACT 


This study proposes to carry out a systematic review of the international scientific production on quality indicators and 
metrics of patents related to universities between the years of 1945 and 2018. The database consulted was the Social 
Sciences Citation Index (Web of Science). The results indicate that 88% of the studies are of a quantitative nature, and the 
most used metric to evaluate the quality of patents is the subsequent citation of patents. Others metrics were indicated, 
such as a) The relative Citation Index (RCI), b) Scientific productivity of the inventor, c) Previous patent citation, d) 
Number of patents accepted, e) Number of patent applications, f) Renewal of patents, g) Patent portfolio. And there are 
still studies that propose the creation of a patent quality index trying to facilitate the measure of patent quality. It is 
observed that there are works with an approach to innovation and the triple helix, which means, the union of the university 
with government and industry, with indicators of commercialization metrics to evaluate the quality of patents in 


universities. 
KEYWORDS: Patent Quality, Performance, Innovation. 
I. INTRODUCTION 


The universities have an important role in the dynamics of forming knowledge and the process of technologic innovation. 
It is so meaningful that since 1970, governments around the world have been releasing countless initiatives, with the goal 
of approximate colleges and the industry (MOWERY and SAMPAT, 2005). It highlights still, that the knowledge 
produced by universities may result in marketable processes and products, benefiting society and the industry. And the 
publication of scientific articles, patents deposits, in such a way, that they serve as protection from discovery to the 
application, divulgation and diffusion of knowledge, given through the transference of technology from universities to 
companies. Once the innovative process has as its ultimate aim, the positive exploration of this knowledge, either in an 
economic approach or using encouragement and improvement of the quality of the research (CARO, LUCIA and 
GARCIA, 2003, HAASE, ARAUJO and DIAS, 2005, BARBASTEFANO, SOUZA and ARAUJO, 2010, PERUCHI e 
MUELLER, 2014, CATIVELLI and LUCAS, 2016) 


Universities contribute the economic growth. Not only through teaching and research, but also through 
engagement and collaboration outside the university environment one way to measure the economic effects of these 


activities is to examine patent data. Over the last years, it has been seen an increase in the use of patent metrics to evaluate 


the innovation and performance of research, and in the last instance, patent quality. (OECD, 2018) There is an impact 
caused by patents originated in universities, as in the adhesion of stimulus through teaching activities and oriented research 
towards innovations and the repercussions for licensing and funding of colleges. (CARO, LUCIA and GRACIA, 2003; 
HAASE, ARAUJO and DIAS, 2005). 


The innovation is considered a primordial factor in the development of organizations, which can occur in any 
sector of economy. Including government services such as health and education an innovation is a continuous process, 
which only occurs when there is the presence of an idea, followed by implementation and results. (OECD, 2015; 


DALLACORTE and JACOSKI 2016) 


Technological innovation is the incorporation of this conception, it is the conception of new technological 
knowledge, whither a new product or a manufacturing process, as well as the attachment of new functionalities or features, 
and that implies incremental improvements with gains of productivity and quality to the productive activities, resulting in 
greater market competitiveness. It is the invention being affectively applied in practice, in another words, it is the 
introduction of new products on the market, or alterations in known products, from on advance in technological 
knowledge. According to WIPO (2017), invention can be considered as a new solution to a specific technological field. 
The innovation can be referred to products and productive processes, the implementation of a product (material assets or 
services) being new or significantly improved, an existing process, a new marketing method, a new organizational method 
in business practices, workplace organization or external relations that changes the operational relations that changes the 
operational way. It perceives that invention is characterized by “conceiving” and innovations by using”. (OCDE, 2015, 
OECD, 2018) 


Since the 1950's, the researchers have discussed the connection between invention, innovation and patents. 
Maclaurin (1953), Schmookler (1960, 1966, 1975) Where (1) patents and innovations are interrelated within inventions, 
because (2) there are innovations in use arising from inventions; (3) There are patents arising from inventions both unused 


and in use, which (4) relate to innovations as shown in figure 1 below 
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Figure 1: Interconnection between patenting, invention and 
innovation Adopted from Basberg (1987). 


The data from universities deposits of patent can rive important trends, in this important innovative process and 
also relating to patenting activity. Highlighting the role of colleges as an intermediate outcome of innovation activity in 
some specific segments and also the university industry partnership the analysis of these data contributes, albeit 


marginally, to a deeper understanding of the Brazilian university in the national innovation system. “ (POVOA, 2006, p3; 


OECD, 2018) 


It should be noted that universities can present an attractive work environment for researches, professors and those 
in transposing research results to the market demand and as a response to the market demand and as an “Additional 
instrument for the evaluation of research results in the academic environment that, depending on the scientific orientation 
of the respective areas, it facilitates clarification on the effective market value of the scientific output “Thus, it becomes 
relevant to know the number of commercializes patents, indicating “The scientific competence and the knowledge or 
technology advantage of universities.”, besides the addition to indicating the quality of research and commercialized 


patents. (HAASE, DIAS and ARAUJO, 2005) 


The trilateral interaction between university, industry and government is the key to innovation and growth in a 
knowledge- based economy, proposed as the “Triple helix” (ETZKOWITZ and LEYDESDORFF, 2000, ETZKOWITZ, 
2008). And the production of marketable innovations, as results of academic research, has also become a reality in several 
countries, and “More and more governments invest in universities as producers of innovation. “ As pointed out by Perucchi 


and Mueller (2014). 


According to the OECD (2015, 2018) and the Frascati Manual (2015) a patent is a property right over in 
invention, granted by national patent offices. A patent gives it holder a limited — duration monopoly over the exploitation 
of the patented invention in return for disclosure, with the aim of allowing a wider social use of the discovery. 
Increasingly, patent statistics are used in various ways by technology students as indicators of the outcome of invention 
activities, because the number of patents granted to a particular company, institution or country may reflect its 
technological dynamism. Examination of patented technologies can give some indication of the direction of technological 
change and is probably the most used. Scientific Liberatore on the determinants and impacts of innovative activity 
increasingly uses patent data at the aggregation (national scale) or company level, due to the close relationship between 
patents and innovation output, which is widely recognized. Patent data also make it possible to identify changes in the 
structure and evolution of inventive activity in countries, industries, companies and technology, by mapping changes in 


technological depend, it’s diffusion and deepness. 


However, there may be some problems with just using patents as indicators: (1) Many innovations do not 
correspond to patented inventions; (2) Many patents correspond to inventions of almost zero technological and economic 
value; although many others never result in innovation, which represents an express challenge between commercial value 
and technological impact, requiring a proposal for evaluating the quality of patents that assesses the invention - innovation 


relationship (BOEING and MUELLER, 2016) 


It is observed that research on the quality of patents can be a relevant contribution to improving the invention- 
innovation relationship. Furthermore, developing ways to assess their quality can enhance the results of academic research. 
Thus, the reflection and question are highlighted: What are the patent quality indicators for universities and their potential 
for generating technological innovation according to the databases of Web of Science? Therefore, the proposal is to present 
a systematic review of the literature on the evaluation indicators of the quality of patents produced by universities based on 
the databases of Web of Science. In an in-depth analysis, what are the features in which Patent Indicators have been 
implemented? This objective helps researchers learn about the usefulness of Patent Indicators. Systematic reviews may 


assist researchers in pinpointing future trends in the field. 


It is considered as specific objects of this paper: 1) Map the articles that approach the subject quality of patents in 
universities; 2) Identify the articles that present metrics to measure the quality of university patents; 3) Organize and 
indicate all these metrics; 4) Indicate the methods used to develop research in the national environment, in order to serve as 
a theoretical and practical framework for improving the evaluative and innovative capacity of universities and their 


research. 


Il. REVIEW OF THE LITERATURE AND METHODS 


Step 1 
Survey in DataBases 


Step 2 
Analysis and selection 
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Figure 2: Article survey and selection process. 
Hl. SYSTEMATIC REVIEW, DATA ANALYSIS AND RESULTS 


In order to present the results in the review, two moments of analysis were established: The first one is intended to 
highlight the general characteristics of the productions: Name of the work, name of the authors, name of the journal 
(source), year of publication, total of quotes. In the second moment, the articles' contribution to understanding the purpose 
of this review are listed, highlighting the patent quality metrics, the research methods used and the contributions of each 


article. 


The survey made it possible to access 298 papers in the Web of Sciences database. After delimitation and 
categorization in the following areas with the respective quantities: Management (N=72); Economics (N=41); Education 
Educational Research (N=23); Information Science Library Science (N=23); Business (N=19); Computer Science 
Interdisciplinary Applications (N=17); Operations Research Management Science (N=11); Computer Science Information 
Systems (N=9); Education Scientific Disciplines (N=7); Business Finance (N=5); Public Administration (N=4); Law 
(N=2) resulted in 158 articles; Selecting only articles and reviews, we reached the number of 120 works. With the reading 
of the abstracts and application of the inclusion and exclusion criteria, 95 were excluded and 25 articles were chosen to 
compose the analysis of this review. The main reasons for exclusion were: (a) distancing from the thematic focus of this 
review, as studies were found that dealt with issues related to Universities, but unrelated to the quality of research, 
specifically patents; (b) articles that addressed data related to the number of patents, without considering their quality or 


even dealing with geographic and demographic issues, unrelated to the focus of the study. 


Referring to the 25 chosen articles, Figure 3 presents an evolution over time of the publications, showing a trend 


of increasing publications over time, with an average of 2.166 articles per year, where the year 2017 peaks in publications, 


with 5 articles. 
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Table | presents a systematization of the information regarding (1) title of the paper, (2) authors, (3) journal, (4) 
year of publication and (5) number of citations. It was observed that the publications on the theme of the study began after 
the year 2001, with a greater concentration from the year 2010, with 20 published articles, that is, 77% of the publications 
occurred after the year 2010. However, most of them refer to previous works, which represent 61.6% of the total citations 
received. It is notable that some journals have a preponderance in relation to the theme, as is the case of the journal 
Research Policy, which has 7 publications, and the Journal of Technology Transfer, which has 2 publications, representing 


38% of the total number of selected articles. These publications still have 51% of the total number of citations received. 


The journal Research Policy (192, 159, 54, 27, 22, 9, 8) has 42.1% of the citations received, Technovation (122) 


received 12% and the International Journal of Industrial Organization (101) received 10% of the citations. 


The most prominent authors are Mowery, DC; Ziedonis, AA, receiving 28.8% of the citations, Owen-Smith, J; 
Powell, WW, with 15.6% and Rasmussen, E; Moen, O; Gulbrandsen, M, with 12% of the citations. It is worth pointing out 
the work of the authors Fini, Riccardo; Lacetera, Nicola; Shane, Scott, from the year 2017 and received 54 citations, which 


represents 5.3% of the citations, which indicates great potential for contribution to the literature. 


Table 2: Summary of the Result of the Selection of Articles 
Tite Avtors Source PY TT 
Academic pateet quality and quantity before and after the Baye-Dole act in the 
United Sates 
The expsading role of uaiersity patenting in the life sciences: assessing the 
empormace of experience and connectnty 
Tetiatives 1 promo commercialization of uanersity knowledze 


Caaages in university patent quality after the Bayb-Dole act: a re-examination 


aaa Er aaa Jaffe, AB: Lereee 3 a 
Taside oc omtside the IP system? Business creation im academia Fini, R; Lacetera, N; Shane, 5 Research Policy 
University research, ieliectsal propesty rights and european innovation ve. 5 Jounal of Economic 
seas — Sueveys 
A Dynamic Network Measure of Technological Change Fonk, RI.; Owea-Sesh, J Masagemes: Sceace 
ae ie ST 
The Effects of Academic Iecubators on University Innovation Kolymp iris, C; Klein, PG. Swategic — 
To Ows, of sotto Owe? A mphilevel analysis of imeliectual property sight «= Hailtlem, N; Amara, N; Olmos- Poy 
policies on academic eawepreneurship Peasels, J; Mohiuddian, M i 
Paveet quality and ownership: An analysis of UR facuhy pavatiag Secu V Research Policy 


The Effects Of Gender On The Quality Of University Patsats And Public 


Research Ceawes In Andabss ia: Is It Bees Wath A Female Presence? bs x : si & Social gy 


Collaborative netmocks and patent production in Andean Comamaay of AC z Review E spanchs de ae 
Natioas universities (UCAN 5), 2005-2015 = Documenucin Cremtifxa 
Paveat swaveay 2 Chinese vaiversities: a comparative perspective Luae, C; Zion, CLV A Sciemomesiks 
. Pn : a P Fach, CO; Block, JH: Journal of Technolo sy 
Chinese university patents: quantaty, quality, and the cole of subs dy programs PG. T. 


A comextaware reseascher recommendation system for univers y-an Garry Wang, Q; Ma, J:Liao,X:Da, W Decision S Sy 


collaboration on RAD projects 
The Sukes in Bayh-Doke: Public Values Beyoad te Pace of Ianovation Vaiéwa, WD. Maen 
Co-omnes relations hips conducive to high quality joiet pateats Eeinss, K Research Policy 
Spatial pameras of moovation activities in France: market's role versus poblic ; Acaals of Regional 
Corsatea, TD ; Jayet. H = 
Follow the (ladus try) Money - The Impact of Sc ence Netmocks and Indus xy- 
to-Universty Conwacts on Academie Paveating in Nanowechaology and Beauty, C ; Ramana, R Iadusry and Innovation 
Biwechnolosy 
Differences in science based innovation by technology Efe cycles: the case of Bs Ieternational Journal of 
sola cell echnolosy Mathes, ; Yeucenaen, T Technolosy Manage mext 
Spatial differences mm the quality of university pateating: Do regions mater? crag imap Research Policy 
Whats te causal effect of RAD on patenting activity a professor's . . Senali Bosieess 
penvdege counry? Evidence from Smeden — Economics 


The sole of universities a the national maovation systems of Chins and the . Asian Journal of 
East Asian NIEs: As exploratory analysis of publications and patenting dasa Saogh, A; Wong, P-E; Ho, ¥ -P 


Table 3 presents a list of articles that presented contributions related to the theme "patent quality metrics", and the 
methods used with the intention of fulfilling the objective of the research. Of the 25 analyzed, it was possible to identify 
quality metrics in 22 articles, that is, 88% of the selected articles presented some type of metric for evaluating patent 
quality. As for the metrics indicated by the articles, the majority use the "forward citation" (17) metric to evaluate patent 
quality, i.e. 82.5% of the total number of articles selected for reading. There are other indications, such as a) Relative 
Citation Index (RCD, b) Scientific productivity of the inventor, c) Backward patent citation, d) Granted Patent, e) Patent 
applications, f) Patent renewals, g) Patent portfolio. There are also authors (MOTOHASHI et al, 2016) who propose the 


creation of a patent quality index to try to facilitate the measurement of patent quality. 


Table 3: Patent Indicators and Metrics Indicated in the Papers 


fusthors , PY 
Wang, O Ma, I, Lao, X; Du, WIDOT 


Passat Qrsality Ind icatoes 
Forsa cdc ao zs backward Fare. tons 


Haltiem N; Amora, N; (lmos-Bensels, Ja; Mobis din, M2017 Mummies of spis-off companies 
Eelympin, Chriws, Kiem, Pew G.. 2017 Forward citations 
Fonk, RL; Ores Seth, J, 2017 a 
Foch, CO; Block, J] H.; Sandoer, PG. 2015 Forward chadogs 
Motohashi, HE; Tomozawa, T. 2016 Forward chadiogs 
Ejenmo, 0 ; Kaliktrom, J, 2016 Fereard cesoe: 
Evizes, KR 2012 Forward citations 
Sangh, A; Woop, P-E Ho, Y-P, 2012 Forward citations The Relative Citation Index (RCT) 
Coma, TD; yes H, 2014 Scismce Hnkage 
Beasdey, 0; Hassan BR, 2013 Forward citations and Patent acplication 
sani sala Fcc atid sill Poaceae 
Puoomta, ME Concemado, Di: AL Maines, ML TOT Forward chadiogs 
Nishieewa, J; Chamero, H, 2011 Forward citations and Patent application 
Fini, KR; Lacetera, N; Shane, 5 , 2010 Scisece Enkase 
Luan, C; Zbos, C; Lin, A, 2010 Pasa: H-index 
Werspages, B. 2005 Forward citations and Publications 
Rass sen, E; Moen, 0; Gulbrandsen, MT (Coen meac is Tees then 
Spe BN; Mone. DC; Zao, A 2 rarer ary TOS 
Crres- Smith, J; Bowe 1, WW , 2008 Forward citations, Pasent Boetfitdlio 
Momery, DO; Ziedonis, AA, 2000 Forward citations 
Jaffe, AB; Lerner, 7, 2001 Forward citations, Patemt Portftdio 


IV. CONCLUSIONS 


Considering the research question of this work about what are the quality indicators of patents used by Universities and 
their potential to generate technological innovation according to the databases of Web of Science journals, with the 
proposal to present a systematic review of the literature of indicators for evaluating the quality of patents produced by 
universities, it was found that the most used indicator by the authors of the selected articles is the subsequent citation of 
patents (JAFFE and LENNER, 2001; MOWERY and ZIEDONIS, 2002, OWNER-SMITH and POWELL, 2003 , 
SAMPAT et al, 2003; RASMUSSEN et al, 2006 , VERPAGEN, 2006; NISHIMURA and OKAMURO, 2010; ACOSTA et 
al, 2012; STERZI, 2013; BEAUDRY and KANANIAN, 2013; SINGH et al, 2015; BRIGGS, 2015; EJERMO and 
KALLSTROM, 2016; MOTOHASHI and TOMOZAWA, 2016; FISCH et al, 2016; KOLYMPIRIS and KLEIN, 2017; 
WANG et al, 2017). Although this metric is not considered the only and definitive one, also because other metrics are 
indicated, such as The Relative Citation Index (RCI), Inventor scientific productivity, Previous patent citation, Number of 
accepted patents, Number of patent applications, Renewal of patents, Patent Portfolio, and even proposals for the creation 
of a patent indicator (MOTOHASHI et al, 2016), it must be considered that it is a relevant indicator and to be considered as 
a metric for assessing the quality of patents. As a research environment, universities have some peculiarities, highlighting 


indications of the inventor's scientific productivity (STERZI, 2013), such as metrics for evaluating the quality of patents, as 


well as the Correlation between the number of patents and published articles (CORSATEA and JAYET, 2014). It is also 
worth highlighting the concern with innovation and the Triple Helix, in the union of the University with the Government 
and Industry, with indications of commercialization metrics (RASMUSSEN and GULBRANDSEN, 2006), formal 
commercialization and spin off (HALILEM et al, 2017). As an indication for future studies, it is worth evaluating the 
visitation in other bases to provide new knowledge and expand the possibility of new indicators. Additionally, the analysis 


of indicators applied in the market to be adapted in universities can be evaluated. 
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